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PREFACE 


The work described in this report was performed by the Observational Systems 
Division of the Jet Propulsion Laboratory. 
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1. INTRODUCTION 


1.1 BACKGROUND 

The directors of the Johnson Space Center and the Jet Propulsion Laboratory 
met during 1977 to explore areas of joint effort that might be beneficial to both 
institutions. One area that was identified as a potential area for cooperative 
effort was image processing. Discussions were initiated between Mr. Gene Rice, 
head of the Earth Resources Program Office (ERPO) at JSC, and Mr. William Green, 
manager of the Science Data Analysis Section at JPL, which includes the Image 
Processing Laboratory (IPL). After a series of meetings between JSC and JPL 
personnel, the activity described in this report was defined and funded, and the 
task was initiated at JPL in March 1978. The study was performed during the 
seven month period from. March to September 1978, at a total cost of $50,000. The 
funding allocation also provided IPL support to a JSC/Mitre Corp. study of future 
computing requirements for NASA's Earth Resources Program. 

At JPL, the task was monitored by Jim Soha, supervisor of the Geology 
Applications Group, and at JSC, by Jim Derbonne of the Earth Resources Program 
Office. Technical coordination at JSC was provided by Don Hay, Jim Sulester, and 
John Lyons, 

Timely performance on this task was possible because of the support provided 
by IPL's color laser film recording facility, under the direction of Mike Wolf, and 
by the JPL Photo Lab managed by John Hewitt. 

1.2 PURPOSE AND OBJECTIVES 

The purpose of this study was to determine if multispectral image processing 
techniques developed at the Jet Propulsion Laboratory's Image Processing Laboratory 
(IPL) for mineral exploration applications could be beneficially applied to the 
analysis of LACIE agricultural data. Landsat multispectral imagery provides the 
main source of data for both applications. 

In the past year, a decorrelation enhancement procedure employing the 
principal component transformation has been successfully used at JPL to increase 
substantially the discriminability of geologic formations in color composites of 
Landsat imagery. The transformation is employed to introduce greater color into 
scenes by more fully utilizing the range of the color domain. A complete descrip- 
tion of the decorrelation procedure and its application to the enhancement of 


1 



color differences is given in the next section. 

The main thrust of this study was to utilize the decorrelation procedure 
in the enhancement of LACIE data. At the outset, the main issue to be addressed 
was whether or not the introduction of more color into LACIE imagery by this tech- 
nique increased the discrimination between wheat and non-wheat areas. The analysis 
logically led to the identification of the following additional issues for evalu- 
ation: 

1. The degree of color consistency remaining between acquisitions 
of a given test site after the process of increasing the color 
range via the decorrelation technique. 

2. The effect of the intrinsic noise in the data on the end pro- 
ducts and the development of techniques to effectively deal 
with the noise. 

3. The potential for multicrop analysis resulting from greater 
color discriminabil ity. 

Section Three defines the various processing techniques employed in the attempt 
to deal with the above issues. Section Four discusses the motivation behind each 
of the processing techniques and gives a preliminary evaluation of the resulting 
products. 

1.3 OVERVIEW OF PROCESSING 

The data base used in this study was made up of 12 LACIE test sites (seg- 
ments). Each segment was composed of from 4 to 13 separate acquisitions. The 
total data base is summarized in Table 1.1. The total number of acquisitions for 
the entire data base was 92. Occasionally, within a segment, a pair of acquisi- 
tions were obtained such that they differed in time by only one day. In this 
study only one acquisition of such a pair was used in the analysis. The remain- 
ing acquisitions (81 in all) formed the working data set. The omitted acquisi- 
tions are followed by an asterisk in Table 1.1. Each segment is characterized 
by (1) LACIE segment number, (2) location by latitude and longitude, (3) verbal 
description of location, e.g., western Kansas, and (4) a list of acquisitions. 

Each acquisition is denoted by a five digit number code. The first two digits 
represent the year in which the acquisition was made. The last three digits 
indicate the Julian day. 

Virtually all processing involved adapting the decorrelation enhancement 
technique to the LACIE application. Several variations of the method were 


2 



investigated. Both linear and non-linear enhancement techniques were applied. 

The use of filtering to reduce the effects of random noise was explored. Normal- 
ization of images to deal with the effects of solar illumination angle and atmos- 
pheric path length was also investigated. 

Generally, a particular processing technique was first applied to a few 
selected individual acquisitions. If the results seemed promising, the technique 
was then applied to a larger portion of the working data set - typically a segment 
or two. Certain techniques were eventually applied to all acquisitions of the 
working data set to permit a comprehensive evaluation. Table 3.3 in Section Three 
summarizes the extent of the processing performed in this study. 

Photographic prints of all processed products are presented in a supplemen- 
tal volume. Copies of the supplement exist at the Earth Resources Program Office 
and the office of Don Hay at JSC, and at the Image Processing Laboratory at JPL 
[contact Jim Soha, (213)354-2722]. A very limited number of examples are repro- 
duced in' this volume. 
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TABLE 1.1 DATA BASE 


SEGMENT 1000 




N040-34/W102-54 


Northeastern Colorado 

76255 76310 

76328 

76363 

77052 

77142 77159* 

77160 

77177* 

77178 

77196 77213* 

77214 



SEGMENT 1183 




N037-34/W095-27 


Southeastern Kansas 

76285 76340 

76358 

77028 

77046 

77063* 77064 

77081* 

77082 

77099 

77118 77153 

77154* 



SEGMENT 1531 




N048-14/W108-16 

■ 

Northern Montana 

76279 76315 

77112 

77129 

77147* 

77148 77184 

77220 



SEGMENT 1537 


• 


N047-27/W105-30 


Western 

Montana 

76349 77091 

77127 

77181 


SEGMENT 1606 




N048-16/W101-22 


Northern N. Dakota 

77125 77143 

77179 

77197 

77250 

SEGMENT 1625 




N047-16/W102-33 


Western 

N. Dakota 

77125 77143 

77179 

77197 

77233 

SEGMENT 1637 




N047-15/W099-19 


Central 

N. Dakota 

77123 77140 

77159 

77194 

77248 
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SEGMENT 1648 


N046-04/ W103-06 


Southwestern N. Dakota 

77107 • 77125 

77143 

77179 


SEGMENT 1734 




N048-19/W1 10-42 


Northern 

Montana 

76245 76263 

77113 77203 

76281 

77059 

77095 

SEGMENT 1739 




N047-45/ Wlll-30 


Northern 

Montana 

76281* 76282 

77132 77149* 

77113* 

77150 

77114 

77168 

77131* 

77222 

SEGMENT 9930 




N037-15/W098-30 


Southern 

Kansas 

76287 76306 
77029 77048 
77191 

• 76323 
77066 

76360 

77083 

77011 

77156 

SEGMENT 9931 




N048-32/W109-19 


Northern 

Montana 

76316 . 76352 

77184 77220 

77058 

77112 

77148 


*acquisitions omitted from working data set. 
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2. DECORRELATION ENHANCEMENT TECHNIQUES 


The basic objective chosen for this task was to introduce greater color 
variation into processed LACIE segments, hopefully aiding in the discrimination 
of crop information. The underlying reason for the rather poor range of color 
content found in many Landsat color composites is the high degree of correlation 
between the image data sensed in the Landsat spectral bands. The standard approach 

4 

of applying contrast enhancements to each of the spectral bands prior to producing 
a color display does not remove this interband correlation. Figure 2.1 illustrates 
schematically, in terms of a 3-dimensional histogram, why correlation between 
bands limits color range. Each image element in a Landsat color composite corres- 
ponds to a point in histogram space determined by the blue, green, and red coordi- 
nate values {in most cases, blue, green, and red correspond to Landsat channels 
4, 5, and 7 respectively). Interband correlation forces most image elements to 
lie in a long narrow region of histogram space near the achromatic axis. Even 
after contrast enhancement of individual components, there are substantial 
regions of histogram space which are underutilized. These underutilized areas 
represent unused color range. Figure 2.2 provides a 2-dimensional illustration 
of the problem for a sample LACIE segment. 

Fortunately a procedure exists for removing correlation from multi spectral 
image components. The principal component, or Karhunen-Loeve, transformation 
has seen widespread use in remote sensing applications. Briefly, each. picture 
element of an n component multispectral scene is regarded as a sample vector from 
an n-dimensional random process. An nxn covariance matrix C for the process is 
estimated [C=E{(x-E{x})(x-E{x})^} where x is a multispectral image vector and 
E{.} denotes expectation] using a subset of image elements. Then an n-dimensional 
rotation which diagonalizes the covariance matrix is applied to each picture 
element in the scene. Since the new covariance matrix is diagonal, no correlation ' 
exists between the new, rotated components. The rotation matrix is composed of 
the n eigenvectors of the original covariance matrix. Any three of the resulting 
principal components can be individually contrast enhanced and then color compos- 
ited for display. Since the individual components are uncorrelated, such a dis- 
play will generally evidence a full range of color. 

The principal components, however, are not the only set of coordinate axes 
available for obtaining a transformed set of uncorrelated image components. If 
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Figure 2.1(a) Correlation between components 
causes the 3-dimens1onal histogram of a typical 
color image to assume an elongated shape, 
generally centered near the achromatic axis. 


Figure 2.1(b) Contrast enhancement of 
individual components enlarges the distribution, 
but underutilized regions representing unused 
color range remain, since correlation is still 
present. 



Figure 2.2(a) A 2-dimensional frequency of Figure 2.2(b) Similar histogram after 

occurrence histogram (log population proportional processing to produce LACIE product 1. 

to brightness) for MSS4 vs MSS5 of unprocessed 
LACIE segment 1739, acquisition 76282. 
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Figure 2.2(c) Regions of unused color are 
readily visible in histogram of LACIE product 3. 


Figure 2.2(d) Correlation remains apparent 
even after the application of a non-linear 
contrast enhancement to individual components 



Figure 2.2(e) Decorrelation enhancement with 
linear variance equalization produces fuller 
utilization of color range. 


Figure 2.2(f) Histogram after decorrelation 
enhancement with Gaussian variance 
equalization. 


ORIGINAL 

OF POOR OU^^LITY 






the principal component images are each contrast enhanced such that the resulting 
variances of the components are all equal, then any further rotation can be 
applied without introducing correlation (Ref. 1). Thus uncorrelated image com- 
ponents can be produced for any possible set of coordinate axes simply by perform- 
ing a principal component transformation, contrast enhancing each component to 
equalize their variances, and then rotating to the desired coordinates. Assuming 
that the variance is appropriately chosen, a color composite display of components 
processed in this manner will include a wide range of color. 

One coordinate system is preferable for the LACIE application, namely that 
system formed by the original image components. These coordinates can be re- 
attained in the decorrelation procedure by following the variance equalization 
contrast enhancement by a rotation which is the inverse of the principal component 
rotation which was used. The primary reason for favoring this return rotation 
is that it allows the basic color relationships of the original scene to be pre- 
served. Thus while absolute colors may sometimes be changed, an area that was 
redder than average, for example, in the original scene will remain redder than 
average after enhancement. This property can be particularly valuable to LACIE 
analysts, who are already attuned to the color relationships inherent in the 
current LACIE color products. 

The contrast enhancement applied to the principal components to equalize 
their variance can be either linear or non-linear. Probably the most effective 
non-linear transformation is a mapping which forces the histogram of each princi- 
pal component to be Gaussian (within quantization limitations). The resulting 
n-dimensional histogram will be a spherically symmetric Gaussian distribution 
(Ref. 2) as will any 3-dimensional subspace histogram. The relative advantages 
of either approach more or less parallel those for black and white image enhance- 
ment. The Gaussian approach forces a more balanced distribution of color, and 
hence is more likely to show differences existing in the input data. The linear 
technique on the other hand tends to preserve biases which exist in the input 
distribution. These biases (e.g., a predominance of one crop type) may be ' 
significant, and would be distorted by the Gaussian approach. To permit full 
evaluation of the decorrelation enhancement technique, both approaches were 
applied to the LACIE test data. The effectiveness of either technique in remov- 
ing interband correlation is demonstrated in the 2-dimensional histograms of 
Figure 2.2. 
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information which is more absolute in meaning and less relative to average scene 
properties. Such an approach of course required that all scenes be normalized 
to remove the effects of all external variables, such as solar illumination angle, 
atmospheric path length, and atmospheric haze. In practice such normalization 
is difficult to perform since adequate information is not available. The results 
of our experiments with global transformations and scene normalization are 
described in Section 4. 

Another significant consideration in applying the decorrelation enhancement 
procedure is the presence of noise. In the principal component transformation, 
total scene variance (summed over all components) is unchanged by the rotation 
(Ref. 5). Since scene information (variance) is concentrated into the higher 
order principal component(s) , the lower order component(s) will naturally contain 
less variance than the original bands. Noise, on the other hand, should be un- 
correlated and hence will remain equally distributed among the components after 
the principal component rotation. Table 2.2 illustrates the situation by listing 
approximate signal -to-noise ratios (SNRs) for the principal components in the 
3-dimensional transformation applied to four acquisitions of segment 1739 (SNRs 
are given as ratios of standard deviations, not variances). An approximate value 
for noise variance in each case was obtained by assuming that the fourth component 
in a 4-dimensional principal component transformation is composed only of noise, 
which is not actually true. The relatively poorer signal to noise ratios of the 
lower order components, particularly for V3 (the third principal component) is 
apparent (the basically noisy character of Landsat data as seen by the low average 
signal to noise ratio of the original components, is also apparent). Now as the 
variance of all components is equalized in the decorrelation enhancement process, 
the noise level in the third component will be increased by a much greater margin 
than will the noise level of the first component (the applied gain is inversely 
proportional to the SNR). In fact it can be seen that the noise (and, of course, 
the signal) in both components 2 and 3 will be increased by a greater amount than 
would have been the case were the original channels enhanced individually (as in 
the case for LACIE products 1 and 3), The resulting higher noise level, due 
particularly to the strong gain applied to the third component, appears as color 
speckle in the enhanced product. The same effect was noted by Juday when he 
used Kauth components to enhance color (Ref. 4). Inspection of the third princi- 
pal component reveals that there is meaningful low and intermediate frequency 
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data in the image. The approach used in this study to preserve the discriminating 
capability of this information while reducing the confusing influence of noise 
was to smooth the third, and sometimes the second, principal component by low 
pass filtering. While simple box filtering (averaging of nearby elements) was 
used in some cases, best results were obtained using a two dimensional low pass 
median filter (Ref, 6). This filter offers the capability of smoothing within 
a field while leaving the field boundary nearly intact. Details of the filtering 
procedures can be found in Section 4, and in Appendix 6.1. Before proceeding 
with a discussion of techniques and results, a summary of the products produced 
for this study will be presented. 
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3, DESCRIPTION OF PROCE’sSING TECHNIQUES 

Thisi section defines the various enhancement techniques used to process the 
imagery in this study. For identification purposes each technique was assigned 
a- two digit product code (PRC) number. This number is found beneath each image 
following the letters PRC. Table 3.2 summarizes the 24 product types produced 
during'the course of this study. Terms which are used in the product codes de- 
scription of Table 3.2 are defined in Table 3.1. 

Products 04-25 all utilize forward and reverse principal component trans- 
formations. Unless otherwise specified, the statistics used in generating a 
particular principal component transformation will have been obtained from the 
individual acquisition under consideration. The forward transformation generates 
principal components V.. These components are then processed (filtered, enhanced 
to equalize variances, etc.) and the reverse transformation applied. This pro- 
duces components B . . As shown in Section 2, the reverse transformation domain 
coincides with the original input domain. The components are then displayed 
as primary colors. 

Table 3.3 summarizes the extent of the application of the various techniques 
to the working data set. 
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TABLE 3.1 Definition of Terms 


FPCT Forward principal component transformation 

RPCT Reverse principal component transformation 


VI 1st principal component 
V2 2nd principal component 
V3 3rd principal component 
V4 4th principal component 


B1 1st reverse transformation component 
B2 2nd reverse transformation component 
B3 3rd reverse transformation component 
B4 4th reverse transformation component 


GVE Gaussian variance equalization 

LVE Linear variance equalization 


Ro The real number R followed by the letter a describes the 
nature -of the variance equalization method used. Its 
meaning is twofold: 

In the case of linear variance equalization (LVE), the 
real number R specifies the number of standard deviations 
on either side of the mean of the distribution that remain 
unsaturated after the linear contrast enhancement is per- 
formed to fix the variance. 

In the case of Gaussian variance equalization (GVE), the 
real number R specifies the extent of the reference 
Gaussian distribution. The distribution being processed 
is then matched to the truncated Gaussian distribution by 
matching cumulative distribution functions, hence fixing 
• the variance. 
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TABLE 3.2 Product Codes Description 


PRC INPUT BANDS PROCESSING DESCRIPTION 


01 MSS 4,5,7 LACIE product 1 

03 MSS 4,5,7 LACIE product 3 

04 MSS 4, 5, 6, 7 FPCT based on statistics for all acquisitions in the given 

segment- GVE 2.3a of V1-V4. RPCT. Bl=blue; B2=green; 
B4=red. 

05 MSS 4, 5, 6, 7 FPCT based on statistics for all acquisitions for the entire 

working data set. GVE 2.3o of V1-V4. RPCT. Bl=blue; 
B2=green; B4=red. 

06 MSS 4, 5, 6, 7 FPCT based on statistics for all acquisitions in the given 

segment. V3 low pass filtered with a 3x3 box of equal 

weights. V4 low pass filtered with a 7x7 box of equal 

weights. GVE 2.3^ of V1-V4. RPCT. Bl=blue; B2=green; 

B4=red. 

07 MSS 4, 5, 6, 7 FPCT based on statistics for all acquisitions for the entire 

working data set. V3 low pass filtered with a 3x3 box of 
equal weights. V4 low pass filtered with a 7x7 box of equal 
weights. GVE 2.3cr of V1-V4. RPCT. Bl=blue; B2=green; B4= 

red. 

08 MSS 4, 5, 6, 7 FPCT. V3 low pass filtered with a 3x3 box of equal weights. 

GVE 2.3a of V1-V3. V4 was omitted in the reverse transfor- 
mation. RPCT. Bl=blue; B2=green; B4=red. 

09 MSS 4,5,7 FPCT. V3 low pass filtered with a 3x3 box of equal weights. 

GVE 2.3aof V1-V3. RPCT. Bl=blue; B2=green; B3=red. 

10 MSS 4,5,7 FPCT. V3 filtered with a 3x3 median filter (see Appendix 6.1) 

GVE 2.3a RPCT. Bl=blue; B2=green, B3=red. 

11 MSS 4,5,7 FPCT. V3 filtered with a 5x5 median filter (see Appendix 

6.1). GVE 2.30 of V1-V3. RPCT. Bl=blue; B2=green; B3=red. 

12 MSS 4,5,7 FPCT. V3 filtered with a 3x3 mode filter (see Appendix 6.1). 

GVE 2.3a of V1-V3. RPCT. Bl=blue; B2=green; B3=red. 

13 MSS 4,5,7 FPCTJ V3 filtered with a 5x5 mode filter (see Appendix 6.1). 

GVE 2 .. 3a of V1-V3. RPCT- Bl=blue; B2=green; B3=red. 

14 MSS 4,5,7 FPCT. V3 low pass filtered with the following 3x3 box: 

1 1 1 
12 1 
111 
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15 MSS 4,5,7 


16 MSS 4,5,7 


17 MSS 4,5,7 


18 MSS 4,5,7 

19 MSS 4,5,7 


20 MSS 4,5,7 


21 MSS 4,5,7 


GVE 2.3(t of V1-V3. RPCT. Bl=blue; B2=green; B3=red. 

FPCT, V3 low pass filtered with the following 5x5 box: 

11111 
1 2 2 2 1 
1 2 2 2 1 
1 2 2 2 1 
11111 

GVE 2.3a of V1-V3. RPCT. Bl=blue; B2=green; 83=red. 

FPCT. V3 low pass filtered with the following 5x5 box: 

11111 

11111 

11111 

11111 

11111 

GVE 2.30 of V1-V3. RPCT. Bl=blue; B2=green; B3=red. 

FPCT. V3 low pass filtered with the following 7x7 box: 

1111111 
1 2 2 2 2 2 1 

1 2 3 3 3 2 1 

1 2 3 4 3 2 1 

1 2 3 3 3 2 1 

1 2 2 2 2 2 1 

1111111 

GVE 2.30 of V1-V3. RPCT. Bl=blue; B2=green; B3=red. 

FPCT. V3 low pass filtered with a 3x3 box of equal weights. 
LVE 2.O0 of V1-V3. RPCT. Bl=blue; B2=green; B3=red. 

FPCT. V2 filtered with a 3x3 median filter; V3 filtered 
with a 5x5 median filter (see Appendix 6.1). LVE 2.O0 
of V1-V3. RPCT. Bl=blue; B2=green; B3=red. 

Input bands first adjusted for sunangle and atmospheric 
effects (see Appendix 6.2). FPCT based on statistics for 
all acquisitions in the given segment. V2 filtered with a 
3x3 median filter; V3 filtered with a 5x5 median filter 
(see Appendix 6.1). GVE 2.3a of V1-V3. RPCT. Bl=blue; 
B2-green; B3=red. 

Input bands first adjusted for sunangle and atmospheric 
effects (see Appendix 6.2). FPCT based on statistics for all 
acquisitions in the given segment. V2 filtered with a 3x3 
median filter; V3 filtered with a 5x5 median filter (see 
Appendix 6.1). GVE 1.0a of V1-V3. RPCT. Bl=blue; B2=green; 
B3=red . 
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22 

MSS 4,5,7 

FPCT. GVE 2.30 of V1-V3. RPCT. 
Bl=blue; B2=green; B3=red. 

23 

MSS 4,5,7 

FPCT. LVE 2.00 of V1-V3. RPCT. 
Bl=blue; B2=green; B3=red. 

24 

MSS 4,5,7 

Input bands first multiplied by a binary mask to exclude 
non-agricultural regions. FPCT based on included regions 


only. GVE 2.30 of V1-V3.. RPCT. 

Bl=blue; B2=green; B3=red. 

25 MSS 4,5,7 Input bands first multiplied by a binary mask to exclude 

non-agri cultural regions. FPCT based on included regions 
only. GVE 2.30 of V1-V3. V2 and V3 filtered with a 3x3 
median filter (see Appendix 6.1). RPCT. Bl=blue; B2=green 
B3=red. 
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TABLE 3.3 PROCESSING SUMMARY 


SEGMENT 

NUMBER 

1 

3 

4 5 

1000 

X 

X 


1183 

X 

X 


1531 

X 

X 


1537 

X 

X 


1606 

X 

X 


1625 

X 

X 

X -X 

1637 

X 

X 

X X 

1648 

X 

X 


1734 

X 

X 


1739 

X 

X 


9930 

X 

X 


9931 

X 

X 



Product Code (PRC) 

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 


X X 
X X 
X X 
X X 
X X 
X X X X 
X X X X 
X X 

XXXXXXXX 
X X 
X X 
X X 


X X 
X X 

XX XXX 

X X 
X X 
X X 

XX XXX 

X X 

X X X X X X • 
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4. DISCUSSION AND EVALUATION 

The purpose of this section is to provide a preliminary evaluation of the 
various processing techniques utilized in this study. The intent here is to make 
observations based on the IPL's experience in multispectral image enhancement. 

Final analysis and evaluation will rest with JSC personnel. 

As a first step, IPL equivalents of LACIE products 1 and 3 (Ref. 7 Section 
2) were created for all acquisitions of the working data set. These were desig- 
nated as products 01 and 03 respectively and were intended to provide a direct 
comparison for subsequent products generated at the IPL. The generation of pro- 
ducts 01 and 03 also eliminated the problem of attempting to compare products 
generated at the IPL with those generated at JSC. All imagery was displayed as 
positive prints rather than positive transparencies in order to facilitate the 
comparison of large numbers of images without requiring the use of a Tight table. 

Figures 4.1 and 4.2 display products 01 and 03 for four acquisitions of 
segment 1531. The 10 x 10 pixel grid was superimposed on all products generated 
in this study so as to reference the pixel to the upper left of a given grid inter- 
section in a manner consistent with the standard LACIE grid system (Ref. 7 p. 4.6) 
Consequently, the pixels immediately to the upper right, lower right, and lower 
left of the grid intersection are partially overwritten by the grid lines in 
order to display the referenced pixel in its entirety. 

As stated in Section 3, processing techniques 04-25 all utilize the decor- 
relation enhancement technique, employing forward and reverse principal component 
transformations. In Section 2, it was shown that the process of variance equali- 
zation in the principal component domain tends to accentuate the inherent uncor- 
related noise in the data producing a color speckle in the end product. Since 
the solution of this problem is basic to the success of the method, the noise 
issue will be discussed first. 

Products 22 and 23 were created solely to demonstrate the accentuation of 
color speckle as a result of the principal component transformation color enhance- 
ment technique. To illustrate the problem, figure 4.3 displays product 23 for 
the same acquisitions of segment 1531 as above. Product 22 possessed virtually 
the same noise characteristics as product 23 and will not be included here. 

While there is obviously more color in product 23 as compared to products 01 and 
03, there is also substantially more color speckle. The problem was to develop 
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techniques which preserve the feature color boundaries {e.g., field boundaries) 
and at the same time, suppress the extraneous color speckle. 

As shown in Section 2, the signal to noise ratio decreases dramatically in 
the lower order principal components. For this reason, the speckle problem was 
attacked by performing noise reduction operations in the principal component 
domain before transforming back to the original input domain. In this study, 
various filters were applied to the lower order principal components in an effort 
to suppress the color speckle in the end products. 

Figure 4.4 displays' product 18 for the same acquisitions as above. In this 
product, the third principal component was filtered with a 3x3 low pass box fil- 
ter before the reverse transformation was applied. When compared to the corres- 
ponding product 23, it is apparent that the color speckle has been noticeably 
reduced. The next experiment then was to try different types of filters on the 
third principal component. In order to evaluate the extent of any field boundary 
degradation, acquisitions from segment 1734 were chosen to be the subject of this 
processing because of the multitude of well defined field boundaries . A total 
of eight different filters (see Section 3) was applied to the third principal 
component before the reverse transformation was performed, creating products 10- 
17. There were no readily visible differences in noise content between any of 
these products and the corresponding product 18, so the imagery is not included 
here. All of these products are included in the photoproducts supplement. 

Inspection of the individual principal components for product 23 for seg- 
ment 1531 revealed that the second principal component in some of the acquisitions 
possessed noticeable speckle. At this point, interest arose over the prospect 
of filtering two of the three principal components. Such a procedure would reduce 
the color speckle, but it might also blur the edge features in the process. 
However, the two dimensional median filter (See Appendix 6.1) used in products 
10 and 11 has the property of preserving edge features and, at the same time, 
suppressing speckle noise. This gave rise to product 19, wherein the third 
principal component was filtered with a 5x5 median filter and the second princi- 
pal component was filtered with a 3x3 median filter prior to the reverse trans- 
formation. Figure 4.5 exhibits product 19 for the same acquisitions displayed 
previously. There is a noticeable improvement over product 18, and upon close 
inspection, the color speckle noise level can be favorably compared with the 
corresponding product 01, while at the same time, the color range is dramatically 
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increased. In addition, the feature boundaries have been preserved. These re- 
sults appeared to be favorable, so products 18 and 19 were made for all acquisi- 
tions of the working data set, and are included for evaluation in the supplemental 
photoproducts volume. 

A given product was often intended to serve as a basis for evaluation of 
more than one processing technique. For example, products 18 and 19 were also 
created to evaluate the effects on color content of linear variance equalization 
as opposed to the Gaussian variance equalization employed in other products. 

With this in mind, the next topic for discussion is the nature of the color con- 
tent produced by the various decorrelation techniques investigated in this study. 

Figures 4.6 and 4.7 display products 01 and 03 for four acquisitions of 
segment 1606. Figures 4.8 and 4.9 display the corresponding products 08 and 09 
for the same acquisitions. Chronologically, products 08 and 09 were the first 
color products created at the IPL by generating the principal component statistics 
using the individual acquisition under consideration. In order to evaluate the 
general promise of the technique, products 08 and 09 were made for all acquisitions 
of the working data set. The results show a dramatic increase in the color range, 
as can be seen by comparing figures 4.6 - 4.9. Aside from a first pass at acqui- 
sition specific color enhancement, products 08 and 09 also provide a comparison 
between the input of MSS 4, 5, 6, and 7, and the input of MSS 4, 5, and 7 into 
the principal component transformation. Even though it is well known that there 
is a very high degree of correlation between MSS 6 and MSS 7, it remained to be 
seen if the addition of MSS 6 to MSS 4, 5, and 7 in the decorrelation enhancement 
process would yield additional information in the color end product. Gaussian 
variance equalization of the principal components was employed in both products, 
matching distribution functions (histograms) to a truncated Gaussian distribution 
of ±2. 3a. There appear to be no significant differences in color information 
between these two products, as shown in figures 4.8 and 4.9. Since processing 
four input bands instead of three obviously requires more time, subsequent pro- 
ducts were generated from MSS 4, 5, and 7 only. 

Next, the choice of the variance equalization method used in the principal 
component domain will be discussed. As pointed out in Section 2, the nature of 
the Gaussian variance equalization method can alter signal relationships in the 
principal component domain in a non-linear fashion. This can affect both the 
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noise and the color characteristics of the end products. To evaluate the relative 
merits of the two variance equalization methods, product 18 was created for a 
direct comparison to product 09. Product 09 employs Gaussian variance equalization 
while product 18 employs linear variance equalization: Figure 4.10 displays the 

corresponding product 18 for the same acquisitions of segment 1606 as above. In 
this example, there appears to be more color definition in the product 18 version. 
In particular, the diagonal feature running through the lower left portion of the 
image seems to be more distinctly separable from its immediate surroundings in 
product 18 than in product 09. It is important to note however that if the initial 
distributions of the principal components closely resemble Gaussian distributions, 
then the two variance equalization methods used in products 09 and 18 will yield 
very similar results. This effect can be seen in figures 4.14 and 4.15. Since 
product 19 showed reasonable promise as discussed previously, its results for the 
segment 1606 acquisitions are also included here, in figure 4.11, to afford the 
reader another opportunity to evaluate the effects of the two dimensional median 
filters applied to the principal components. Prints of products 18 and 19 for 
the entire test data base are included in the supplement. 

The next issue to be addressed is the color consistency between acquisitions 
for the various products using the principal component transformation. An exami- 
nation of figures 4.6 to 4.11 shows that the various colors present in products 
01 and 03 generally remain readily identifiable in products 08, 09, 18, and 19 
but in a very accentuated form. That is, a given color will retain its basic 
character and not be transformed to an entirely different color. Slight color 
shifts may occur, usually involving unsaturated colors in the products 01 and 03. 
However, basic color relationships are preserved in the enhancement process, so 
that enhanced colors possess a clear interpretive compatibility with current LACIE 
products. Color trends visible in a sequence of acquisitions in products 01 and 
03 are also readily apparent after decorrelation enhancement. This basic color 
compatibility with current LACIE products, including similarity of variation 
across acquisitions, was a key design goal in the implementation of the enhance- 
ment process, and can be verified for the entire test data set by inspection of 
the photoproducts supplement. 

Up to this point, all products discussed have been processed using acquisi- 
tion specific statistics, as is the case with standard LACIE products. As shown 
above, the products utilizing some form of the decorrelation enhancement process 
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possess a greater color range than the corresponding products 01 and 03. For 
this reason, processing techniques were investigated which utilize the increased • 
color range to maintain strict color consistency over an arbitrary selection of 
acquisitions. By strict color consistency, it is meant that a given input n-tuple 
of pixel values will produce the identical color in the end product, regardless 
of the acquisition in which it is found. To accomplish this, for each of the 
input bands {e.g., MSS 4, 5, and 7), a merged data set was created which contained 
the data for that band from all the acquisitions under consideration. The princi- 
pal component transformation and the corresponding variance equalization parameters 
were generated using the statistics from the merged data sets. Each individual 
acquisition could then be processed using a fixed principal component transforma- 
tion and variance equalization parameters, ensuring that the resulting color in 
the end products was consistent in all the acquisitions. Consequently, the color 
expansion was not optimal for any of the individual acquisitions but rather 
represented optimal processing for all the acquisitions taken together. Applying 
this process to the original test data produced product 04 which utilized statistics 
generated from all acquisitions in a given segment, and product 05 which utilized 
statistics generated from all the acquisitions in the entire working data set. 
Products 06 and 07 are filtered versions of products 04 and 05 respectively and 
do not differ in color content from them- 

Not surprisingly, inadequate results were obtained when composite statistics 
gathered from sets of unnormal ized’ input images were used. The individual acqui- 
sition data clusters in histogram space are somewhat segregated within the com- 
posite histogram, so that the color contrast in products 04 and 05 is low for all 
the individual acquisitions. The problem arises from the fact that variations 
in the acquisitions can be due both to real changes in the scene and to external 
influences which do not pertain to crop status. While the procedure aims to pre- 
serve the first category of variations, the second category is a significant noise 
factor which can introduce severe errors in interpreting real change. Thus 
ideally, all external influences would first be removed by normalizing the input 
scenes to "standard" viewing conditions. 

Two external influences, namely variations in solar illumination angle and 
in atmospheric path length, can be- easily modelled and removed. A detailed de- 
scription of how this normalization was performed is given in Appendix 6.2. To 
obtain a color consistent decorrelation enhancement, then, individual acquisitions 
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are normalized before composite statistics are accumulated. Products 20 and 21 
represent the results of this technique as applied to the generation of a segment 
specific principal component transformation and variance equalization parameters. 
Note that these two products differ only in the extent of the reference Gaussian 
distribution used in the variance equalization (±2.3o for product 20 and ±1 .Oa 
for product 21). In addition, both products 20 and 21 employ two dimensional 
median filters in the principal component domain. To compare the end results. 
Figures 4.12 - 4.15 display products 01, 03, 09, and 19 for a sequence of four 
of the six acquisitions for segment 1739. Figures 4.16 and 4.17 display the cor- 
responding products 20 and 21. Comparison of products 20 and 21 with products 01 
03, 09, and 19 reveals a noticeable increase in the color range over products 01 
and 03 but, as was expected, not as much of an increase as with products 09 and 

19. There does seem to be slightly better contrast in product 21 than in product 

20. This can be attributed to the more uniform distributions obtained as a result 
of the variance equalization utilizing a +I.O 0 truncated Gaussian distribution 
instead of one of ±2.3cr. One would expect greater color speckle to accompany a 
harsher variance equalization. It appears however, as though the two dimensional 
median filters have successfully suppressed the speckle while retaining the slight 
increase in basic color differences. Finally, as a result of enforcing color 
consistency in products 20 and 21, the red color of the strip fields appears to 
increase in intensity as the growing season progresses. In products 01, 03, 09, 
and 19, the color of the strip fields remains basically the same. Products 20 
and 21 then, may for this segment, more accurately reflect the crop maturation 
process. The degree to which the composite distribution (in histogram space) is 
tightened, relative to the distributions of individual acquisitions, provides a 
tool for assessing the impact of normalization. Details of the assessment pro- 
cedure, together with results for several test segments, are provided in Appendix 
6.3. Unfortunately, this processing technique -did not yield results which were 
uniformly as promising as those shown here when applied to other segments. One 
significant external variable, atmospheric haze, was not removed, since its degree 
of influence is in practice not easily determined. The remaining variation among 
acquisitions is probably due to some combination of legitimate changes in scene 
spectral response and variations in atmospheric conditions, with possibly some 
other unidentified variables present. The normalization procedure does appear to 
be helpful in some cases. Further study might suggest heuristic procedures, for 
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example based on MSS4 mean value, which would allow haze effects to be adequately 
reduced. 

Near the end of this study, products 24 and 25 were created in order to 
evaluate the effect of generating the principal component transformation based 
on the agricultural regions only. The intent was to attempt to force more color 
into regions of particular interest. This was done by first constructing a 
binary mask of a given segment using available ground truth. This mask contained 
a 1 where there was agriculture and a 0 everywhere else. Each band image of an 
acquisition within the segment was then multiplied by the mask image, leaving 
agricultural regions undisturbed and mapping non-agricultural regions to 0. 

The principal component transformation color enhancement technique was then 
applied to these images. For reasons which are as yet not fully understood, the 
color expansion was no niore pronounced than with other products utilizing the 
principal component transformation based in the complete image. Since the masking 
process required forehand knowledge of the ground utilization, the technique was 
not studied further and the images created will not be shown here. 
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5. SUMMARY AND CONCLUSION 


For comparison purposes, IPL equivalents of LACIE products 1 and 3 were 
created for all acquisitions of the working data set. Twenty-two separate new 
products were created, all of which utilized forward and reverse principal com- 
ponent transformations. The problem of accentuated color speckle seemed best 
dealt with by performing two dimensional median filtering on the lower order 
principal components before the reverse transformation was applied. There appeared 
to be little difference in the end products between the input of MSS 4-, 5, 6, and 
7, and MSS 4, 5, and 7 into the principal component transformation color enhance- 
ment process, so MSS 4, 5, and 7 were used almost exclusively. There also appeared 
to be little difference between the two types of variance equalization utilized 
in the principal component domain. Where there were differences, the linear 
variance equalization method seemed better than the Gaussian variance equaliza- 
tion method. 

Although the principal component transformation color enhancement technique 
dramatically increased the color range, individual colors retained their basic 
character and were not transformed to totally different regions of the color 
spectrum. Strict color consistency coupled with color expansion was enforced over 
all the acquisitions within a given segment through the use of merged data sets 
in the generation of the principal component transformation and variance equali- 
zation parameters. The initial processing produced color products of low contrast 
which was attributed to the disjointedness of the individual acquisition data 
histograms. A sun angle and atmospheric path length correction was applied to 
the individual acquisitions in an attempt to tighten up the merged data set dis- 
tributions before the principal component transformation was performed. The end 
products seemed to show that while some segments yielded promising results, the 
individual acquisition data histograms were still too disjoint in other segments. 
Unnormalized variations due to atmospheric haze remained, and may be the source 
of the difficulty. 

In general, greater coTor discriminabil ity could be obtained through the 
use of the principal component transformation color enhancement process. Such 
processing seems to show promise, especially in terms of multicrop analysis where 
the greater obtainable color range may make the distinction between different 
crops more pronounced. 
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6. APPENDICES 


6.1 TWO DIMENSIONAL MEDIAN AND MODE FILTERS 

The two dimensional median filter is an nxm box filter which replaces the 
center element of the box with the median of the distribution of all the elements 
in the box. For example, if a 3x3 median filter is being performed and the ele- 
ments in -the box have the following values: 

2 1 2 

' 3 3 3 

4 5 4 

then the element in the center will remain a 3. If the element in the lower 
right corner were changed to a 10, the element in the center would still remain 
a 3 after the filter was- performed. In this way, high frequency information is 
removed while low frequency information is preserved. However, if a feature edge 
is encountered, the median value of the box will change abruptly as the box moves 
across the edge, hence preserving the high frequency information related to the 
edge. In this manner, the high frequency information related to speckle can be 
suppressed while that relating to feature edges can be preserved. 

The two dimensional mode filter is an nxm box filter which replaces the 
center element of the box with the mode of the distribution of all elements in 
the box. If a 3x3 mode filter is being applied to the following box: 

7 2 4 
1 1 6 

5 8 1 

then the center element would not be changed. The corresponding median filter 
would change the center element to a 4. 

6.2 SUNANGLE AND ATMOSPHERIC PATH LENGTH CORRECTIONS 

To correct for differences in solar illumination angle and corresponding 
atmospheric path length, the data under consideration was multiplied by a factor 
which normalized the effective incident radiation to a sunangle of 90° and hence 
an atmospheric path length of one. The corrections for sunangle and atmospheric 
path length were determined separately and then multiplied together to give the 
. ..._tota.l correction factor. The assumption made in this correction was that the 
observed brightness is directly proportional to the incident radiance. To 
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FIGURE 6.2.1 Solar illumination geometry 



FIGURE 6.2.2 Atmospheric path geometry 


63 



TABLE 6.2.1 


SUNANGLE AND ATMOSPHERIC PATH 
LENGTH CORRECTIONS FOR SEGMENT 1739 





SUNANGLE 

PATH LENGTH 

TOTAL 

ACQUISITION 

MSS 

SUNANGLE 

CORRECTION 

CORRECTION 

CORRECTION 

76282 

4 

31 

1.94 

1.09 

2.11 


5 

31 

1.94 

1.05 

2.04 


6 

31 

1.94 

1.03 

2.00 


7 

31 

1.94 

1.01 

1.96 

77114 

4 

46 

1.39 

1.04 

1.45 


5 

46 

1.39 

1.02 

1.42 


6 

46 

1.39 

1.01 

1.40 


7 

46 

1.39 

1.01 

1.40 

77132 

4 

51 

1.29 

1.03 

1.33 


5 

51 

1.29 

1.01 

1.30 


6 

51 

1.29 

1.01 

1.30 


7 

51 

1.29 

1.00 

1.29 

77150 

4 

53 

1.25 

1.02 

1.28 


5 

53 

1.25 

1.01 

1.26 


6 

53 

1.25 

1.01 

1.26 


7 

53 

1.25 

1.00 

1.25 

77168 

4 

54 

1.24 

1.02 

1.26 


5 

54 

1.24 

1.01 

1.25 


6 

54 

1.24 

1.01 

1.25 


7 

54 

1.24 

1.00 

1.24 

77222 

4 

47 

1.37 

1.03 

1.41 


5 

47 

1.37 

1.02 

1.40 


6 

47 

1.37 

1.01 

1.38 


7 

47 

1.37 

1.00 

1.37 
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determine the sunangle correction consider the diagram of Figure 6.2.1. If 8 is 
the solar elevation angle and is the solar radiance, then the actual energy 
per unit area radiating the surface is 1^ sin 6. Therefore to correct to a sun- 
angle of 90°, the observed brightness values need to be multiplied by 1/sin e. 

From Kondratyev (Ref. 8, p. 162), the atmospheric path length can be used 
to calculate the atmospheric mass m by taking the ratio of the path length at 
the given sunangle to the path length in the vertical direction as shown in the 
Figure 6.2.2. If the top of the atmosphere is AA' (assumed , to be flat) then 
m = 1/sin 0. Using this result and a table of energy dis'tributions in the solar 
spectrum for different atmospheric mass (Ref. 8, p. 222), a correction could be 
determined to normalize the effective path length to a sunangle of 90°. This 
was done for each of MSS 4, 5, 6, 7. 

The two correction terms were multiplied together to give the total correc- 
tion factor. Table 6.2.1 lists the correction factors applied to all the acqui- 
sitions for segment' 1739. 


6.3 ASSESSMENT OF NORMALIZATION RESULTS 

The degree to which the composite distribution (in histogram space) is 

tightened, relative to the distributions of the individual acquisitions, provides 

an approach for evaluating the success of the normalization. To perform the 

assessment, the means of all of the bands for each of the acquisitions in the 

given segment may be examined before and after the normalization is applied. In 

order to represent the distribution of these means, a mean and standard deviation 

of the means themselves was calculated for each band. Let p-. be the mean for 

1 K 

band i of acquisition k. Then 


m. = 


^ 1 
n 


n 

£ 

k=l 


'ik 


1 




(ut, 



where n is the number of acquisitions in the given segment. Since all of the 
correction factors are greater than unity, the distribution of the post-normaliz- 
ation means may have a larger variance than the pre-normalization means, but in 
fact be relatively closer together. To account for this, a measure of the rela- 
tive change in the distribution of the acquisition means may be obtained by cal- 
culating a relative standard deviation by taking the ratio (s^./m. ) before and 
after normalization. Further, to make the effects of normalization more readily 
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apparent, the ratio of the relative standard deviations may be calculated as 
follows: 

I fi I Post- 
^ m, j normal i zation 


pre- 
normalization 

Therefore, if is less than 1, the distribution of acquisition means in band i 
has been tightened as a result of the normalization process. 

Table 6.3.1 displays the above quantities for all the segments of the 
working data set except segments 1531, 9930 and 9931. (The solar elevation 
values for these three segments were incorrect.) Using segment 1739 as a rough 
guide, inspection of the table shows that only about half of the segments showed 
promise using this technique. -One may attribute part of the lack of complete 
success to other external influences such as atmospheric haze, whose effects are 
more difficult to determine. On the other hand, if the acquisition data distri- 
butions are in reality sufficiently disjointed, then the modelling performed here 
should not be expected to merge the individual distributions. When this is the 
case, a subset of the acquisitions under consideration might be chosen to yield 
better results. 
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TABLE 6.3.1 


RELATIVE 

CHANGE 

IN THE DISTRIBUTION 

OF ACQUISITION MEANS 

AS A RESULT 

OF SUNANGLE AND ATMOSPHERIC 

PATH LENGTH 

NORMALIZATION 

SEGMENT 


PRE-NORM. 

POST- NORM. 


NO. 

BAND 

s./m. 

s^/m^. 


1000 

4 

0.35 

0.12 

0.34 


5 

0.34 

0.12 

0.35 


6 

0.35 

0.08 

0.23 


7 

0.31 

0.05 

0.16 

1183 

4 

0.28 

0.14 

0.50 


5 

0.23 

0.20 

0.87 


6 

0.43 

0.15 

0.35 


7 

0.39 

0.12 

0.31 

1531 

4 

0.22 

0.10 

0.45 


5 

0.21 

0.12 

0.57 


6 

0.27 

0.08 

0.30 


7 

0.26 

0.07 

0.27 

1606 

4 

0.15 

0.10 

0.67 


5 

0.16 

0.13 

0.81 


6 

0.22 

0.15 

0.68 


7 

0.23 

0.16 

0.70 

1625 

4 

0.12 

0.13 

1.08 


5 

0.17 

0.20 

1.18 


6 

0.10 

0.06 

0.60 


7 

0.11 

0.06 

0.55 

1637 

4 

0.16 

0.11 

0.69 


5 

0.18 

0.16 

0.89 


6 

0.23 

0.17 

0.74 


7 

0.27 

0.21 

0,78 

1648 

4 

0.19 

0.18 

0.95 


5 

0.19 

0.18 

0.95 


6 

0.17 

0.13 

0.76 


7 

0.18 

0.13 

0.72 

1734 

4 

0.25 

0.08 

0.32 


5 

0.25 

0.08 

0.32 


6 

0.26 

0.11 

0.42 


7 

0.23 

0.08 

0.35 

1739 

4 

0.15 

0.05 

0.33 


5 

0.13 

0.07 

0.-54 


6 

0.18 

0.06 

0.33 


7 

0.17 

0.05 

0.29 
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